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A  key  tenet  of  the  Better  Buying  Power  initiatives  is  to  increase 
small  business  participation  in  Department  of  Defense 
contracting.  The  department  has  had  mixed  success  in  retaining 
small  businesses  and  meeting  small  business  contracting 
Results  of  a  survey  given  to  681  small  business  leaders  show  many 
factors  commonly  exist  that  prevent  small  businesses  from  pursuing 

defense  contracts.  Some  factors  are  more  common  than  others, 

ivjgfM _ 

with  the  most  cited  factors  related  to  a  lack  of  communication 
from  government  leads  or  to  the  government  taking  too  long  to  give 
approvals  and  make  decisions.  Statistical  evidence  also  supports 
the  perceptions,  of  smaller  and  newer  small  businesses,  that  the 
defense  business  is  more  challenging  for  them  than  for  their  larger 
and  more  experienced  competitors.  However,  this  turned  out  to  be 
the  case  for  only  a  subset  of  the  factors  we  explored. 
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Creating  more  than  half  of  this  country’s  gross  domestic  product  and 
seven  out  of  every  10  new  jobs  (Graves,  n.d.,  p.  1),  small  business  holds  a 
place  of  importance  in  the  U.S.  economy  that  cannot  be  overstated.  Small 
businesses  are  also  a  key  driver  of  innovation,  producing  on  average  13  times 
more  patents  per  employee  than  large  firms  (Mielach,  2012,  para  4). 

Small  businesses  are  also  an  important  contributor  in  defense  acquisition, 
providing  innovation,  competition,  and  services  at  great  value  (Simmers, 
2011).  The  Department  of  Defense  (DoD)  has  recognized  that  it  must  attract 
and  retain  small  businesses  in  order  to  continue  to  create  and  maintain 
world-class  weapon  systems.  Department  leadership  has  pushed  for 
increased  small  business  roles  and  opportunities  through  the  Better  Buying 
Power  initiatives,  seeking  to  provide  “the  maximum  practicable  opportu¬ 
nity”  for  small  businesses  (Kendall,  2015,  p.  25).  The  department  has  also 
set  a  small  business  prime  contracting  goal,  which  has  ranged  from  21  to 
23%  of  all  contract  obligations  per  year  from  2011  through  2016. 


The  DoD  has  had  mixed  success  in  meeting  its  small 
business  prime  contracting  goal.  It  had  missed  this 
goal  for  7  years  in  a  row  before  finally  meeting  it  in 
2014  (Serbu,  2015,  para  1). 


The  DoD  has  had  mixed  success  in  meeting  its  small  business  prime  con¬ 
tracting  goal.  It  had  missed  this  goal  for  7  years  in  a  row  before  finally 
meeting  it  in  2014  (Serbu,  2015,  para  1).  Some  attribute  this  to  the  nature  of 
what  the  department  buys:  aircraft,  tanks,  and  large  weapon  systems  often 
beyond  the  capabilities  of  small  businesses  (Lee,  2012).  However,  critics  of 
the  Pentagon’s  small  business  policies  claim  it  is  too  simple  and  self-cen¬ 
tered  (Chandler,  2014),  focusing  on  a  single  contract  obligation  goal  that 
does  not  take  into  account  the  needs  and  challenges  from  the  perspective  of 
the  small  businesses  themselves.  Businesses  will  respond  to  a  government 
request  for  proposal  (RFP)  only  if  it  presents  a  satisfactory  business  case, 
something  the  government  often  overlooks  (Chandler,  2014).  Attracting 
small  businesses  to  conduct  business  with  DoD  has  been  a  challenge.  An 
analysis  of  small  businesses  in  the  Central  Contractor  Registration  data¬ 
base  showed  that  of  the  small  businesses  conducting  business  with  DoD 
from  1997  to  2007,  44%  had  stopped  conducting  business  with  DoD  in 
2008  (Moore  et  al.,  2008,  p.  85).  Only  1.9%  of  the  businesses  had  continuous 
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contracts  withDoD  over  the  11  years.  Further,  46%  of  small  businesses  that 
received  contracts  during  this  period  received  only  one  contract,  which  was 
valid  for  a  year  or  less,  possibly  indicating  an  unwillingness  of  businesses 
to  pursue  additional  opportunities  after  their  initial  contract  was  complete 
(Moore  et  al.,  2008,  p.  72). 


The  challenges  facing  businesses  in  defense  contracting  have  been  pre¬ 
viously  well  documented.  The  Lexington  Institute  recently  published  an 
article  calling  the  DoD  a  “difficult  buyer”  that  discourages  businesses  from 
submitting  bids  on  defense  contracts  and  proposing  that  industry  and  gov¬ 
ernment  take  more  of  a  partnership  approach  to  contracting  (Chandler, 
2014).  The  Government  Accountability  Office  (GAO)  has  also  published 
multiple  reports  claiming  the  DoD  needs  to  improve  reporting  and  tracking 
of  small  business  financial  obligations  (Neumann,  2015;  Shear,  2011;  Woods, 
2013).  Most  recently,  the  challenges  related  to  small  businesses  have  got¬ 
ten  the  attention  of  Congress,  and  the  House  Armed  Services  Committee 
(HASC)  directed  a  panel  to  investigate  challenges  currently  affecting  the 
defense  industry  base. 


Factors  Affecting  Small  Business  Participation 
in  DoD  Contracts 

We  conducted  a  comprehensive  literature  review  to 
better  understand  the  factors  preventing  small  busi¬ 
nesses  from  desiring  to  conduct  business  with  DoD.  This 
review  included  previous  panel  interviews  of  business 
owners,  including  the  2012  HASC  study;  opin¬ 
ionated  articles;  case  studies;  and  GAO 
reports.  The  purpose  of  this  review 
was  not  to  determine  whether  the  per¬ 
ceptions  presented  in  these  articles 
were  valid,  but  simply  to  identify 
what  factors  exist.  We  found  26 
factors  that  are  potential  bar¬ 
riers  preventing  businesses 
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from  pursuing  an  initial  contract  or  follow-on  contracts  with  DoD.  The 
factors  we  found  fit  into  three  categories:  the  contract  solicitation  and  award 
process,  contract  requirements,  and  contract  execution.  A  summary  of  the 
barriers  we  found  in  each  category  is  provided  below. 

The  Contract  Solicitation  and  Award  Process 

The  contract  solicitation  documentation  and  the  metrics  for  evaluating 
proposals  were  cited  as  being  too  complex  and  difficult  to  understand  by 
business  leaders  (HASC,  2012).  A  lack  of  communication  during  the  solici¬ 
tation  process  was  also  cited  as  an  issue,  with  businesses  struggling  to  get 
answers  to  their  questions  on  solicitation  documentation  and  desiring  more 
feedback  on  proposals  that  did  not  win  a  contract  (Maser  &  Thompson, 
2013).  Given  the  complexity  of  the  solicitation  documentation,  the  amount 
of  paperwork  required  to  submit  proposals  and  insufficient  time  to  develop 
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proposals  were  also  cited  as  barriers  (Krieger,  2015).  Previous  studies  also 
found  that  delays  in  making  contract  awards  have  created  financial  hard¬ 
ships  on  small  businesses  (Chaplain,  2010). 


The  literature  review  revealed  that  many  small 
businesses  continue  to  struggle  with  defense 
business  after  winning  contracts. 


Contract  Requirements 

Defense  contracts  have  unique  requirements  for  placing  a  bid  that  are 
not  found  in  the  commercial  world.  While  many  of  these  requirements  serve 
a  good  purpose— often  to  protect  the  government— they  were  also  cited  to  be 
barriers  to  defense  business  in  the  articles  we  reviewed.  Contract  require¬ 
ments  that  were  cited  as  being  issues  in  previous  studies  include  surety 
bonds,  government  cost  accounting  standards,  export  control  regulations 
(International  Traffic  in  Arms),  and  a  past  performance  rating  on  previous 
government  contracts  (H  ASC,  2012).  Regulations,  both  the  complexity  and 
the  number  of  them,  were  also  cited  as  an  issue  (Friar,  2015).  Protecting 
proprietary  data  was  another  area  of  concern  for  businesses  in  previous 
case  studies  (Chaplain,  2010).  Negotiation  of  data  rights  has  been  getting  an 
increased  amount  of  attention  in  the  last  few  years,  both  from  industry  and 
the  government  (Erwin,  2014).  Another  issue  with  contract  requirements 
cited  in  previous  studies  is  technical  requirements  being  written  too  nar¬ 
rowly  or  seemingly  catering  to  a  particular  vendor  (Chaplain,  2010),  thus 
preventing  other  businesses  from  reasonably  competing  for  these  solicita¬ 
tions.  Contractor  profits  have  long  been  a  source  of  contention  and  were  also 
cited  as  a  barrier  to  defense  contracts,  because  many  businesses  can  make 
higher  margins  in  the  commercial  sector  (Chandler,  2014). 

Contract  Execution 

The  literature  review  revealed  that  many  small  businesses  continue 
to  struggle  with  defense  business  after  winning  contracts.  Cited  barriers 
associated  with  contract  execution  include  issues  getting  payments  for 
completed  work  or  payments  taking  too  long,  challenges  contacting  govern¬ 
ment  leads  or  government  leads  not  being  helpful  when  contacted,  contract 
modifications  taking  too  long,  and  approvals  taking  too  long  (Krieger,  2015; 
Mills,  2010). 
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Research  Hypothesis  Methodology 

Based  on  our  literature  review,  there  is  no  question  that  barriers  exist 
and  that  they  prevent  small  businesses  from  desiring  to  pursue  defense 
contracts.  What  is  unknown  is  how  common  these  barriers  are.  Are  they 
widespread  issues  or  isolated  incidents?  We  wanted  to  gain  more  insight 
into  these  barriers  and  how  they  affect  small  business  participation  in 
defense  contracts.  Specifically,  we  wanted  to  determine  the  following: 

•  How  common  are  the  previously  identified  barriers  to  defense 
business?  The  prior  works  cited  above  are  either  opinionated 
articles  or  are  studies  conducted  with  a  handful  of  small  busi¬ 
nesses.  A  recent  large-scale  study  with  quantitative  data  on 
these  barriers  does  not  exist. 

•  How  do  these  barriers  vary  by  industry  or  business  type?  DoD 
conducts  business  in  numerous  areas,  and  previous  studies 
have  argued  that  a  rigid  “one  size  fits  all”  contracting  approach 
may  not  be  appropriate  (Blakey,  2011,  p.  4). 

•  Do  new  small  businesses  perceive  any  of  these  potential  bar¬ 
riers  to  be  more  of  an  issue  than  those  that  have  extensive 
experience  conducting  defense  business?  Studies  have  shown 
that  “nontraditional”  small  businesses  with  little  to  no  gov¬ 
ernment  experience  struggle  to  compete  for  defense  contracts 
(Cox,  Moore,  &  Grammich,  2014).  Many  of  these  nontraditional 
small  businesses  have  extensive  commercial  experience  and/ 
or  new  and  innovative  technologies  that  the  department  could 
use  (Freedberg,  2014). 

•  Do  smaller  small  businesses  perceive  any  of  these  barriers  to 
be  more  of  an  issue  than  those  that  are  larger  but  still  qualify  as 
a  “small  business”?  The  qualifications  to  be  considered  a  small 
business  vary  by  industry,  but  are  on  the  order  of  500  to  1,500 
employees.  Previous  studies  have  cited  these  qualifications  as 
too  large  (Bail,  2010),  stating  that  smaller  small  businesses 
with  tens  of  employees  cannot  compete  for  small  business  set- 
asides  against  those  with  hundreds  or  thousands  of  employees. 

To  gain  further  insight  into  our  research  hypothesis,  we  developed  and 
administered  a  survey  for  small  businesses.  A  large-scale  survey  was 
administered  to  small  businesses  that  had  formerly  conducted  business 
with  or  are  currently  conducting  business  with  the  DoD.  A  mailing  list  was 
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developed  from  contract  award  data  and  small  business  registries.  An  elec¬ 
tronic  mail  invitation  was  sent  directly  to  the  business  CEO,  president,  or 
business  development  lead  if  the  information  was  available  (this  informa¬ 
tion  was  available  for  78%  of  the  invitations  that  were  sent).  Otherwise,  the 
invitation  was  sent  to  a  publicly  known  address  accompanied  by  a  request 
that  it  be  forwarded  to  the  appropriate  point  of  contact.  The  survey  was  con¬ 
ducted  from  September  to  November  2015,  and  we  received  681  responses 
from  small  businesses. 


The  first  portion  of  the  survey  consisted  of  collecting  demographic  informa¬ 
tion  about  the  business:  primary  line(s)  of  business,  number  of  employees, 
and  business  history  with  DoD.  The  participants  were  then  given  a  list  of  26 
factors  and  asked  to  rate  the  importance  of  each  factor  in  their  decision  not 
to  pursue  additional  defense  business  opportunities.  The  26  factors  were 
selected  based  on  our  literature  review  of  previous  articles  on  the  subject. 
Each  of  the  factors  was  rated  using  a  5-point  Likert  scale  ranging  from 
“not  a  factor”  to  “very  important  factor.”  The  respondents  were  also  given 
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an  option  to  select  “don’t  know  or  no  opinion”  for  each  factor;  for  analysis 
purposes,  we  discarded  both  these  responses  and  responses  that  were  left 
unanswered.  Participants  were  also  given  three  open-ended  qualitative 
questions  to  express  any  other  potential  barriers  not  considered  in  the 
Likert-scale  questions,  to  provide  examples,  and  to  offer  suggestions  on  how 
to  make  defense  business  more  attractive. 


Demographics 

Survey  respondents  included  businesses  representing  the  major  areas 
in  which  DoD  conducts  business.  Respondents  included  “technical”  ser¬ 
vice-based  businesses  such  as  small  business  innovation  research  (SBIR) 
participants  and  engineering  support  firms,  “nontechnical”  service-based 
businesses  providing  services  DoD  uses  every  day  (such  as  janitorial  and 
grounds  maintenance),  product-based  businesses  such  as  manufacturers 
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and  retailers,  and  construction  businesses.  Many  of  the  respondents  con¬ 
duct  business  with  DoD  in  multiple  areas.  Respondents’  length  of  experience 
conducting  business  with  DoD  spanned  a  wide  range,  from  businesses  rela¬ 
tively  new  to  the  industry  (1-2  years)  to  ones  conducting  business  with  DoD 
for  more  than  20  years  (Figure  1). 


FIGURE  1.  SURVEY  RESPONDENTS:  PRIMARY  LINE  OF  BUSINESS 
AND  LENGTH  OF  BUSINESS  HISTORY 


Primary  Line  of  Business 


Products-  and  Service-Based-Technical 
Products-Based  (Manufacturer,  Retailer,  Wholesaler) 
Construction 

Service-  and  Products-Based— Nontechnical 
Service- Based-Nontechnical 
Service-Based— Both  Technical  and  Nontechnical 
Service-Based-Technical 


Length  of  Business  History  With  DoD 


Less  than  2  Years 
2  to  5  Years 
5  to  10  Years 
10  to  20  Years 
More  than  20  Years 


Respondents  also  included  small  businesses  having  a  fairly  wide  range  in 
size.  Many  respondents  had  only  a  handful  of  employees  working  for  their 
business;  42%  had  10  or  fewer,  and  64%  had  30  or  fewer.  The  large  major¬ 
ity— 87%— had  fewer  than  100  employees.  We  also  received  responses  from 
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the  “larger”  businesses  with  several  hundred  employees  that  do  still  qualify 
for  small  business  contracts,  making  up  13%  of  the  total  survey  respondents 
(Figure  2). 


|  10  or  Fewer  Employees  [  11-30  Employees 

|  31-100  Employees  J  More  than  100  Employees 


Quantitative  Survey  Results 

The  survey  results  showthat  of  the  26  potential  barriers  surveyed,  most 
were  perceived  to  exist  to  a  reasonably  common  extent;  19  of  them  were 
rated  as  a  “somewhat  important”  or  “very  important”  barrier  by  more  than 
half  of  the  respondents  in  their  decision  not  to  pursue  additional  business 
opportunities  with  the  DoD.  A  smaller,  although  fairly  significant,  propor¬ 
tion  of  the  respondents  cited  the  barriers  as  being  a  “non-issue”  or  only 
“very  minor,”  ranging  from  13%  to  half  of  the  respondents.  The  full  results 
are  shown  in  Figure  3. 
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FIGURE  3.  SURVEY  RESULTS 


Survey  Results— All  Responses 


Government  approvals  or  reviews  take  too  long 
Insufficient  feedback  on  bids/proposals  when  did  not  win 
Government  evaluation  of  proposals  takes  too  long 
Profits  too  low 

Government  leads  not  accessible  or  difficult  to  contact 
Too  much  paperwork  required  to  submit  for  proposals 
Guidance/metrics  for  evaluating  proposals  too  complex 
Government  market  is  too  uncertain  or  unstable 
Government  regulations  are  too  complex 
Government  leads  having  poor  attitudes  or  not  helpful 
RFP/bid  documentation  is  too  complex 
Requirements  are  written  too  specific/narrow  to  meet 
Unresponsive  answering  questions  for  proposal 
Contract  documents  are  too  long/complex 
Government  not  responsive  in  resolving  contract  issues 
Contract  modification/change  process  is  too  slow 
Too  much  competition  from  other  firms 
Government  websites  hard  to  use 
Certified  cost  and  pricing  data  difficult  to  obtain 
Government  accounting  standards  bard  to  meet 
Data  rights 

Past  performance  rating  is  hard  to  obtain 
Payments  too  slow 
Export  control  regulations 
Not  enough  time  to  submit  proposals/bids 
Surety  Bonds  hard  to  obtain 


MEM  18%  28% 


— '  20%  25% 


■  Very  Important  Factor  ■  Very  Unimportant  Factor 
Somewhat  Important  Factor  Not  Factor 
|  Somewhat  Unimportant  Factor 
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The  barriers  in  Figure  3  are  ranked  from  top  to  bottom  in  the  order  they 
were  cited  as  being  “very  important”  or  “somewhat  important,”  the  most  to 
the  least.  Four  of  the  five  top-ranked  barriers— approvals  taking  too  long, 
insufficient  feedback  on  proposals,  proposal  evaluations  taking  too  long, 
and  inaccessible  government  leads— are  directly  related  to  a  lack  of  com¬ 
munication  from  government  leads  or  to  the  government  taking  too  long  to 
make  approvals  and  decisions.  Sixty-five  to  70%  of  respondents  rated  these 
as  either  “somewhat  important”  or  “very  important.”  Waiting  for  approvals 
was  the  top-ranked  factor,  with  many  respondents  giving  examples  of  wait¬ 
ing  for  approvals  during  both  the  contract  award  process  and  during 
contract  execution.  One  respondent  provided  an  example  in  which  it  took 
months  to  get  approved  to  place  a  bid  for  parts,  only  to 
get  approval  after  the  solicitation  had  closed. 
Contract  award  times  on  the  order  of  6  months  to 
a  year  for  simple  contracts  were  commonly 
cited  by  the  respondents.  One  respondent 
gave  an  example  of  choosing  to  “no-bid”  a 
phase  II  SBIR  contract  after  winning 
a  phase  I  contract,  explaining  that 
they  could  not  afford  to  keep  the 
J  |  l||  company’s  principal  investiga- 

1^,  |||  tor  on  the  payroll  for  the 

amount  of  time  the  gov¬ 
ernment  takes  to  award 
the  contract.  One 
respondent  questioned 
how  the  government  can 
take  excessive  times  to  award 
contracts  and  give  minimal  time 
for  businesses  to  prepare  proposals 
and  suggested  the  government  be  held 
to  the  same  timeframe  as  they  are.  The 
final  barrier  ranked  in  the  top  five  was  low 
profits,  long  a  source  of  contention,  which  was 
cited  as  the  fourth  most  important  barrier,  with 
64%  of  respondents  naming  it  as  an  important  issue 
preventing  them  from  pursuing  additional  defense 
business  opportunities.  Many  respondents  referred  to 
profit  levels  decreasing  on  government  contracts,  often 
to  “single  digit  levels.”  Respondents  said  that  profit  lev¬ 
els  of  10  to  15%  would  normally  be  acceptable.  However, 
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defense  contracts  require  extensive  overhead  to  develop  proposals  and  meet 
other  contract  requirements,  such  as  accounting  standards,  surety  bonds, 
and  security  clearances.  This  “excessive”  overhead  further  eats  into  profits 
and  makes  profit  levels  of  10  to  15%  unattractive  and  profit  levels  less  than 
this  unsustainable. 


Numerous  respondents  also  felt  that  government 
regulations  favor  larger  businesses,  as  small  busi¬ 
nesses  have  to  comply  with  the  same  regulations,  but 
cannot  hire  proper  personnel  to  understand  them. 


Sixty  to  63%  of  respondents  ranked  three  of  the  remaining  barriers,  among 
the  10  most  cited,  as  “somewhat  important”  or  “very  important.”  These 
are  associated  with  the  large  amount  of  effort  and  specialized  knowledge 
required  by  the  defense  acquisition  process:  too  much  paperwork  required 
to  submit  proposals  (ranked  sixth),  guidance/metrics  for  evaluating  pro¬ 
posals  being  too  complex  (ranked  seventh),  and  government  regulations 
being  too  complex  (ranked  ninth).  Many  respondents  cited  the  need  to  hire 
subject  matter  experts  to  fully  understand  government  solicitations  and 
regulations.  Numerous  respondents  also  expressed  that  government  reg¬ 
ulations  favor  larger  businesses,  as  small  businesses  have  to  comply  with 
the  same  regulations,  but  cannot  hire  proper  personnel  to  understand  them. 
Respondents  cited  costs  for  the  business  and  planning  effort  to  develop 
proposals  on  the  order  of  tens  to  hundreds  of  thousands  of  dollars.  This  is 
significantly  higher  than  commercial  proposal  costs,  leading  them  to  pursue 
commercial  contracts  in  place  of  defense  contracts.  The  government  market 
being  too  unstable  was  ranked  eighth,  with  many  of  the  respondents  not¬ 
ing  that  the  recent  instability  of  budgets  and  contract  awards  has  reduced 
their  ability  to  plan  and  conduct  business  development,  also  further  eating 
into  profits.  Government  personnel  not  being  helpful  when  contacted  was 
ranked  10th. 

Following  the  top  10  cited  barriers,  the  factors  are  largely  focused  on 
the  contract-solicitation-and-award  process  and  issues  with  contract 
documents.  For  the  contract-solicitation-and-award  process:  proposal 
documentation  being  too  complex  was  ranked  11th,  requirements  in  the 
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solicitation  being  written  too  narrowly  to  be  reasonably  met  was  ranked 
12th,  and  the  government  not  being  responsive  in  answering  questions 
during  the  solicitation  period  was  ranked  13th.  Many  respondents  noted 
that  they  had  regularly  noticed  solicitations  put  out  for  competition  that 
were  seemingly  tailored  to  a  particular  vendor,  as  the  solicitation  had  very 
specific  requirements  that  no  other  vendor  could  reasonably  meet.  For 
issues  with  contract  documents:  contract  documents  being  too  long  and 
complex  was  ranked  14th,  the  government  not  being  responsive  in  resolving 
contract  issues  was  ranked  15th,  and  the  contract  modification  process 
being  too  slow  was  ranked  16th.  One  respondent  noted  that  the  slow  pace  of 
contract  and  requirement  modifications  limited  the  number  of  innovations 
they  could  provide  to  the  government. 

Contract  requirements  unique  to  defense  contracts  were  among  the  lowest 
ranked  barriers.  Certified  cost  data,  government  accounting  standards, 
data  rights,  past  performance  ratings,  export  control  regulations,  and 
surety  bonds  were  ranked  19th,  20th,  21st,  22nd,  24th,  and  26th,  respec¬ 
tively.  In  general,  a  minority  of  the  respondents— 31  to  50%— rated  these 
factors  as  “somewhat  important”  or  “very  important.”  This  indicates  that 
although  these  were  cited  in  previous  studies  and  in  this  study,  they  are 
less  widespread  as  barriers  for  small  businesses  than  the  other  barriers 
already  mentioned. 

Differences  by  Industry 

We  divided  the  survey  responses  into  four  independent  groups,  based  on 
primary  line  of  business:  construction,  technical  service-based,  nontech¬ 
nical  service-based,  and  product-based.  For  this  portion  of  the  analysis, 
respondents  that  conducted  business  in  multiple  areas  were  discarded. 

The  Mann-Whitney  and  Kruskal-Wallis  statistical  tests  are  commonly  used 
to  determine  whether  population  medians  are  equal  or  differ  among  inde¬ 
pendent  groups.  The  tests  use  a  ranking  methodology  in  which  all  responses 
for  a  factor  are  ranked  from  largest  to  smallest  and  then  compare  the  mean 
ranking  for  each  independent  group.  Table  1  provides  the  factors  that  dif¬ 
fered  enough  between  industries  to  be  statistically  significant,  based  on 
pairwise  Mann-Whitney  analysis.  The  z  values  for  the  Kruskal-Wallis  test 
are  also  shown.  The  2:  value  helps  to  interpret  how  the  average  rank  for  each 
group  compares  to  the  ranks  from  all  groups.  A  negative  2:  value  indicates 
that  the  particular  group  perceived  the  barrier  to  be  less  important  when 
compared  to  all  groups,  and  a  higher  2:  value  indicates  that  the  particular 
group  perceived  the  barrier  to  be  more  important  when  compared  to  all 
groups.  As  the  absolute  value  of  the  z  value  gets  larger,  the  further  away  a 
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particular  group’s  average  rank  gets  from  the  overall  average  rank.  A  greater 
absolute  value  indicates  that  the  group  perceived  the  barrier  to  be  more  or 
less  of  a  barrier  to  a  greater  extent  when  compared  to  a  lesser  absolute  value. 
Based  on  these  tests,  we  found  that  the  majority  of  the  factors— 21  of  the 
26— presented  no  statistically  significant  results.  The  five  that  did  present 
statistically  significant  results  are  shown  in  Table  1. 


1  TABLE  1.  STATISTICALLY  SIGNIFICANT  DIFFERENCES 

BETWEEN  INDUSTRIES 

Barrier 

Result 

Kruskal-Wallis 
z  value 

Product:  2.80 

Government 
websites  hard 
to  use 

Product-based  small  businesses 
perceive  this  to  be  MORE  of 

Technical:  -1.91 

a  barrier  than  the  other  three 
industries  do. 

Nontechnical:  -1.14 

Construction:  -0.34 

Data  rights: 

government 

requesting 

proprietary 

information 

Technical  service-based  and 

Product:  2.43 

product-based  small  businesses 
perceive  this  to  be  MORE  of  a 

Technical:  -0.27 

barrier  than  nontechnical  service- 
based  and  construction  small 

Nontechnical:  -1.70 

businesses. 

Construction:  -2.33 

Technical  service-based  and 

Product:  0.72 

product-based  small  businesses 

Technical:  1.67 

Export  control 

perceive  this  to  be  MORE  of  a 

regulations 

barrier  than  nontechnical  service- 
based  and  construction  small 

Nontechnical:  -2.26 

businesses. 

Construction:  -2.72 

Product:  -3.36 

Product-based  small  businesses 

Technical:  2.30 

Payment 

perceive  this  to  be  LESS  of  a 

issues 

barrier  than  the  other  three 
industries  do. 

Nontechnical:  0.96 

Construction:  0.69 

Product:  1.59 

Surety  bonds 
being  difficult 
to  obtain 

Technical  service-based  small 
businesses  perceive  this  to  be 

Technical:  -3.76 

LESS  of  a  barrier  than  the  other 
three  industries  do. 

Non-Technical:  1.17 

Construction:  2.84 
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Based  on  the  Mann-Whitney  results,  technical  service-based  and  prod¬ 
uct-based  small  businesses  found  data  rights  and  export  control  regulations 
to  be  more  of  a  barrier  compared  to  those  that  provide  nontechnical  products 
and  services  to  the  DoD.  This  doesn’t  come  as  much  of  a  surprise,  as  these 
two  items  generally  do  not  apply  to  the  other  two  industries.  Technical  ser¬ 
vice-based  small  businesses  have  a  relatively  easier  time  obtaining  surety 
bonds  for  their  contracts.  Small  businesses  that  produce  products  for  DoD 
have  a  relatively  easier  time  getting  paid  for  their  work  but  have  a  harder 
time  finding  opportunities  on  government  websites. 


Experience  as  a  Factor 

We  used  the  Spearman’s  rho  correlation  coefficient  as  a  preliminary 
analysis  of  the  relationship  between  each  of  the  factors  and  length  of  busi¬ 
ness  history  with  DoD.  The  coefficient  measured  the  relationship  between 
each  of  the  factors  and  groups  of  businesses  with  various  levels  of  expe¬ 
rience  (Figure  1),  both  of  which  are  ordinal  data.  Fourteen  of  the  factors 
presented  correlation  coefficients  that  were  statistically  significant.  Of 
these  14  coefficients,  all  but  one  were  negative,  ranging  from  -.099  to  -.263, 
indicating  that  the  factors  become  less  of  a  barrier  as  the  business’s  level  of 
experience  increases.  The  one  factor  that  provided  a  statistically  significant 
positive  trend  was  “government  market  being  too  uncertain  or  unstable,” 
with  a  correlation  coefficient  of  .084. 

We  broke  the  responses  into  independent  groups  by  level  of  experience 
and  again  used  the  Mann-Whitney  and  Kruskal -Wallis  statistical  tests  to 
look  for  differences.  Ten  of  the  26  factors  did  present  statistical  differences 
between  the  groups  based  on  the  Mann-Whitney  pairwise  analysis.  These 
10  factors  and  associated  Kruskal-Wallis  z  values  are  shown  in  Table  2. 


18 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 2-29 


January  2017 


TABLE  2.  STATISTICALLY  SIGNIFICANT  DIFFERENCES  BETWEEN 

LEVELS  OF  DoD  EXPERIENCE 

Barrier 

Result 

Kruskal-Wallis 
z  value 

Government 

The  group  of  businesses  with 
less  than  2  years  of  experience 

<  2  years:  2.06 

2-5  years:  -0.35 

websites  hard 

perceives  this  to  be  MORE  of 
a  barrier  than  the  groups  of 

10-20  and  more  than  20  years  of 
experience. 

5-10  years:  1.33 

to  use 

10-20  years:  -1.53 

>  20  years:  -0.97 

Government 
not  providing 
enough  time 
to  submit 
proposals/bids 

The  group  of  businesses  with 
less  than  2  years  of  experience 

<  2  years:  2.12 

2-5  years:  -0.65 

perceives  this  to  be  MORE  of 
a  barrier  than  the  groups  of 
businesses  with  10-20  and  more 
than  20  years  of  experience. 

5-10  years:  1.43 

10-20  years:  -1.09 

>  20  years:  -1.30 

Insufficient 
feedback  on 

The  group  of  businesses  with 
less  than  2  years  of  experience 

<  2  years:  2.25 

2-5  years:  0.82 

bids/proposals 

perceives  this  to  be  MORE  of 
a  barrier  than  the  groups  of 

10-20  and  more  than  20  years  of 
experience. 

5-10  years:  0.46 

when  failing  to 
win  contract 

10-20  years:  -1.11 

>  20  years:  -1.94 

Request  for 
proposal/ 
quotation/bid 
documentation 

The  group  of  businesses  with 
less  than  2  years  of  experience 
perceives  this  to  be  MORE  of  a 

<  2  years:  2.15 

2-5  years:  -1.25 

5-10  years:  1.19 

is  too  complex 
or  difficult  to 
understand 

barrier  than  the  group  with  more 
than  20  years  of  experience. 

10-20  years:  -0.22 

>  20  years:  -1.44 

Data  rights: 
government 
requesting 

The  groups  of  businesses  with  less 
than  2  and  2-5  years  of  experience 
perceive  this  to  be  LESS  of  a  barrier 

<  2  years:  -0.81 

2-5  years:  -1.94 

5-10  years:  -0.22 

proprietary 

information 

than  the  group  with  more  than  20 
years  of  experience. 

10-20  years:  -0.14 

>  20  years:  2.68 

Contract 
documents  are 

The  group  of  businesses  with 
less  than  2  years  of  experience 

<  2  years:  2.38 

2-5  years:  0.40 

too  long  and 

perceives  this  to  be  MORE  of 
a  barrier  than  the  groups  of 

10-20  and  more  than  20  years  of 
experience. 

5-10  years:  0.28 

complex  to 
understand 

10-20  years:  -1.13 

>  20  years:  -1.32 
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TABLE  2.  STATISTICALLY  SIGNIFICANT  DIFFERENCES  BETWEEN 

LEVELS  OF  DoD  EXPERIENCE,  CONTINUED 

Barrier 

Result 

Kruskal-Wallis 
z  value 

Certified  cost 

The  groups  of  businesses  with  less 
than  2  and  2-5  years  of  experience 
perceive  this  to  be  MORE  of  a 

<  2  years:  2.31 

2-5  years:  1.09 

data  difficult  to 

5-10  years:  -0.03 

provide 

barrier  than  the  group  with  more 
than  20  years  of  experience. 

10-20  years:  0.00 

>  20  years:  -2.66 

Surety  bonds 
being  difficult 

The  group  of  businesses  with 
less  than  2  years  of  experience 
perceives  this  to  be  MORE  of  a 

<  2  years:  4.01 

2-5  years:  -0.02 

5-10  years:  -0.37 

to  obtain 

barrier  than  all  four  groups  with 
more  than  2  years  of  experience. 

10-20  years:  -1.62 

>  20  years:  -1.02 

Past- 

The  groups  of  businesses  with 
less  than  2  and  2-5  years  of 

<  2  years:  3.90 

2-5  years:  2.67 

performance 
rating  difficult 
to  obtain 

experience  perceive  this  to  be 

MORE  of  a  barrier  than  the  groups 
of  10-20  and  more  than  20  years  of 
experience. 

5-10  years:  -0.02 

10-20  years:  -2.85 

>  20  years:  -2.13 

Government 

accounting 

standards 

All  four  groups  of  businesses  with 
less  than  20  years  of  experience 
perceive  this  to  be  MORE  of  a 

<  2  years:  2.34 

2-5  years:  1.10 

5-10  years:  1.93 

difficult  to 
meet 

barrier  than  the  group  with  more 
than  20  years  of  experience. 

10-20  years:  -0.97 

>  20  years:  -3.57 

Small  businesses  with  less  experience  found  nine  of  these  10  factors  to  be 
more  of  a  barrier  than  did  those  businesses  with  more  experience.  The 
pairwise  analysis  we  conducted  showed  that  the  point  at  which  differences 
became  statistically  significant  between  groups  did  vary  by  factor.  Six  of 
the  factors  were  perceived  to  be  more  of  a  challenge  by  those  with  less  than 
two  years  of  experience  when  compared  to  those  with  more  experience; 
two  factors  were  perceived  to  be  more  of  a  challenge  by  all  businesses  with 
less  than  5  years  of  experience  when  compared  to  those  with  more.  All 
businesses  with  less  than  20  years  of  experience  found  government 
accounting  standards  more  difficult  than  those  with  more  than  20  years  of 
defense  experience.  Four  of  these  factors,  which  are  “extra”  requirements 
unique  to  defense  business,  do  not  come  as  much  of  a  surprise:  certified 
cost  data,  government  accounting  standards,  having  satisfactory  past 
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performance  with  the  government,  and  surety  bonds.  Newer  small  busi¬ 
nesses  also  view  the  lack  of  feedback  from  proposals  and  the  complexity  of 
documentation  as  deterrents  to  pursuing  additional  defense  opportunities. 
The  government  requesting  data  rights  or  proprietary  information  was 
perceived  as  less  of  a  barrier  by  small  businesses  with  less  than  5  years  of 
DoD  experience  when  compared  to  those  with  more  than  20  years  of  expe¬ 
rience.  This  is  possibly  because  they  do  not  have  as  much  proprietary 
information  or  they  may  have  more  flexibility  sharing  it  to  get  their  foot  in 
the  door  with  government  contracts. 


Small  businesses  with  extensive  defense  experience 
view  competition  from  others  as  much  of  a  barrier  as 
those  just  breaking  into  the  defense  market. 


Equally  as  important,  many  factors  did  not  statistically  vary  by  level  of 
experience.  Small  businesses  with  extensive  defense  experience  view 
competition  from  others  as  much  of  a  barrier  as  those  just  breaking  into  the 
defense  market.  The  same  holds  true  for  levels  of  profit  and  the  other  factors 
not  mentioned  in  Table  3. 

Size  Qualifications 

We  broke  the  respondents  into  four  independent  groups  based  on  num¬ 
ber  of  employees:  1-10, 11-30,  31-100,  and  more  than  100.  We  again  used 
Spearman’s  rho  correlation  coefficient  as  a  preliminary  analysis  of  the 
relationship  between  the  factors  and  number  of  employees.  This  time  eight 
of  the  26  factors  presented  correlation  coefficients  that  were  statistically 
significant.  Seven  of  these  presented  negative  correlation  coefficients,  rang¬ 
ing  from  -.08  to  -.215,  indicating  a  slight  trend  that  the  factors  become  less 
of  an  issue  as  business  size  increases.  However,  the  trend  is  fairly  weak,  as 
the  majority  of  the  factors  do  not  present  statistically  significant  correlation 
coefficients.  The  one  factor  that  provided  a  statistically  significant  positive 
trend  was  “profits  too  low,”  with  a  correlation  coefficient  of  .142. 

We  again  broke  out  the  respondents  into  four  independent  groups  by  num¬ 
ber  of  employees  and  used  the  Kruskal- Wallis  and  Mann-Whitney  tests  to 
examine  statistically  significant  differences  in  the  median  response.  Eight 
of  the  26  factors  were  statistically  significant  based  on  the  Mann-Whitney 
pairwise  analysis.  These  factors  and  their  associated  Kruskal-Wallis  z 
values  are  shown  in  Table  3. 
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TABLE  3.  STATISTICALLY  SIGNIFICANT  DIFFERENCES 

BY  COMPANY  SIZE 

Barrier 

Result 

Kruskal-Wallis  z  value 

The  group  of  businesses 

1-10  employees:  1.18 

Government 

with  10  or  fewer 
employees  perceives  this 

11-30  employees:  0.20 

websites  hard  to  use 

to  be  MORE  of  a  barrier 
than  the  group  with  more 
than  100  employees. 

31-100  employees:  0.30 

>100  employees:  -2.29 

Request  for 
proposal/quotation/ 
bid  documentation 

All  three  groups  of 

1-10  employees:  0.97 

businesses  with  100 
or  fewer  employees 
perceive  this  to  be  MORE 
of  a  barrier  than  the 
group  with  more  than 

100  employees. 

11-30  employees:  0.33 

is  too  complex 
or  difficult  to 

31-100  employees:  0.52 

understand 

>100  employees:  -2.40 

The  group  of  businesses 
with  10  or  fewer 
employees  perceives  this 

1-10  employees:  -3.35 

Profits  too  low 

11-30  employees:  1.24 

to  be  LESS  of  a  barrier 
than  all  three  groups  with 
more  than  10  employees. 

31-100  employees:  0.93 

>100  employees:  2.18 

Contract  documents 
are  too  long 

The  groups  of  businesses 

1-10  employees:  0.75 

with  1-10  and  11-30 
employees  perceive  this 

11-30  employees:  1.90 

and  complex  to 
understand 

to  be  MORE  of  a  barrier 
than  the  group  with  more 
than  100  employees. 

31-100  employees:  -0.76 

>100  employees:  -2.39 

The  groups  of  businesses 
with  1-10  and  11-30 
employees  perceive  this 
to  be  MORE  of  a  barrier 
than  the  groups  with  31- 
100  and  more  than  100 
employees. 

1-10  employees:  2.51 

Certified  cost  data 

11-30  employees:  1.34 

difficult  to  obtain 

31-100  employees:  -1.94 

>100  employees:  -2.76 

The  groups  of  businesses 
with  1-10  and  11-30 
employees  perceive  this 
to  be  MORE  of  a  barrier 
than  the  groups  with 
31-100  and  more  than 

100  employees. 

1-10  employees:  2.61 

Past-performance 
rating  is  difficult  to 
obtain 

11-30  employees:  1.59 

31-100  employees:  -2.52 

>100  employees:  -2.42 

22 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 2-29 


January  2017 


TABLE  3.  STATISTICALLY  SIGNIFICANT  DIFFERENCES 
BY  COMPANY  SIZE,  CONTINUED 


Barrier  Result  Kruskal-Wallis  z  value 


Government 
accounting 
standards  difficult 
to  meet 


The  groups  of  businesses  mo  employees:  3.74 

with  1-10  and  11-30  - 

employees  perceive  this  11-30  employees:  1.73 

to  be  MORE  of  a  barrier  - 

than  the  groups  with  31-100  employees:  -3.05 

31-100  and  more  than  - 

100  employees.  >100  employees:  -3.60 


Government 
regulations  are 
too  difficult  to 
understand 


The  groups  of  businesses 
with  1-10  and  11-30 
employees  perceive  this 
to  be  MORE  of  a  barrier 
than  the  groups  with 
31-100  and  more  than 
100  employees. 


i  i w  ci  i  ipiuy cw. 


11-30  employees:  2.81 
31-100  employees:  -1.07 
>100  employees:  -2.54 


Smaller  small  businesses  found  seven  of  these  eight  factors  to  be  more  of  a 
barrier  than  larger  ones.  A  pairwise  analysis  again  showed  that  the  point 
at  which  differences  became  statistically  different  did  vary  some  between 
the  factors.  Five  of  the  factors,  including  three  of  the  “extra”  requirements 
specific  to  government  contracts,  were  perceived  to  be  more  of  a  challenge 
by  businesses  with  30  or  fewer  employees  when  compared  to  the  larger 
businesses.  Businesses  with  fewer  than  10  employees  found  government 
websites  and  government  regulations  to  be  more  burdensome  when  com¬ 
pared  to  the  larger  small  businesses  with  more  than  100  employees.  All 
businesses  with  fewer  than  100  employees  perceived  RFP  documentation 
to  be  complex  when  compared  to  businesses  with  more  than  100  employees. 
Businesses  with  fewer  than  10  employees  perceived  profits  (or  lack  thereof) 
as  less  of  a  barrier  to  defense  contracts  when  compared  to  larger  businesses. 
Again,  this  possibly  indicates  that  smaller  businesses  are  more  willing  to 
sacrifice  higher  profits  to  get  in  the  door  with  government  contracts.  We 
found  no  statistical  evidence  to  support  the  idea  that  the  larger  companies 
perceive  competition  from  other  firms  as  less  of  a  barrier  than  those  smaller 
companies  with  only  a  handful  of  employees;  the  same  holds  true  for  the 
other  factors  not  mentioned  in  Table  3. 
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Ideas  for  a  Micro-Business  Category 

Six  factors  statistically  proved  to  be  more  of  a  barrier  by  both  businesses 
with  less  DoD  experience  and  the  smaller  small  businesses:  government 
websites  being  hard  to  use,  RFP  documentation  being  too  complex,  contract 
documents  being  too  complex,  certified  cost  data  being  hard  to  provide, 
past-performance  rating  being  hard  to  obtain,  and  government  accounting 
standards  being  hard  to  meet.  The  creation  of  a  defense  micro-business 
category  that  focuses  on  reducing  these  six  factors  as  barriers  would  likely 
improve  participation  from  smaller  and  nontraditional  small  businesses. 
Based  on  the  survey  results,  limiting  the  size  qualifications  to  approximately 
30  employees  would  be  appropriate.  Ideas  for  implementation  of  this  cate¬ 
gory,  provided  by  survey  respondents,  are  shown  below: 

•  Combine  all  of  the  relevant  small  business  DoD  websites  into 
a  single  user-friendly  one  that  shows  all  steps  needed  to  prop¬ 
erly  develop  and  complete  proposals.  Also  consider  developing 
a  central  website  that  would  help  facilitate  teaming  among 
complementary  small  businesses. 

•  For  technical  proposals,  use  white  papers  as  an  initial  screen¬ 
ing  process.  The  initial  screening  based  solely  on  technical 
merit  will  help  industry  and  government  focus  on  technical 
content  and  avoid  the  preparation  and  review  of  lengthy  pro¬ 
posals,  saving  both  proposers  and  target  agencies  time  and 
resources. 

•  Adopt  commercial  business  practices,  particularly  commercial- 
style  contracts.  Tailor  contracts  to  the  greatest  extent  possible 
by  eliminating  clauses  that  do  not  apply  to  the  particular 
contract. 

•  Ease  up  on  past-performance  requirements  on  these  solici¬ 
tations,  particularly  when  new  technology  or  innovation  is 
important.  Evaluate  previous  commercial  experience  as  an 
alternative  to  looking  only  at  past  defense  contracts  on  these 
solicitations. 

•  Make  requirements  for  government-approved  accounting 
systems  less  burdensome.  Provide  accounting  software  that 
would  allow  small  businesses  to  be  compliant  with  cost-re¬ 
imbursement  contracts  rather  than  having  them  develop  or 
source  their  own  software. 
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Summary  of  Findings 

We  find  that  most  of  the  factors  preventing  small  businesses  from 
participating  in  defense  contracts  identified  in  previous  studies  are 
widely  perceived  to  exist.  Of  the  26  potential  barriers  surveyed,  most 
were  rather  commonly  perceived  to  exist ;  19  of  them  were  rated  as  a 
“somewhat  important”  or  “very  important”  barrier  by  more  than  half  of 
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the  respondents.  Five  of  the  factors  provided  statistically  significant  dif¬ 
ferences  in  perception  between  industries:  government  websites  being  hard 
to  use,  data  rights,  export  control  regulations,  payment  issues,  and  surety 
bonds  being  difficult  to  obtain.  However,  21  of  them  did  not,  indicating  the 
majority  of  the  barriers  we  explored  are  perceived  with  no  significant  dif¬ 
ference  across  industry  types. 

We  did  find  statistical  evidence  to  support  the  idea  that  businesses  with 
less  defense  business  experience  perceive  defense  business  to  be  more 
challenging  than  those  with  extensive  defense  experience.  We  also  found 
support  for  smaller  small  businesses  perceiving  defense  business  to  be  more 
challenging  than  those  larger  businesses  that  still  qualify  for  small  business 
contracts.  However,  this  turned  out  to  be  the  case  for  only  a  minority  of  the 
factors  we  explored:  nine  factors  were  perceived  as  more  of  a  barrier  by 
businesses  with  less  defense  experience,  seven  factors  were  perceived  as 
more  of  a  barrier  by  the  smaller  small  businesses,  and  six  factors  were  per¬ 
ceived  as  more  of  a  barrier  by  both  businesses  with  less  experience  and  the 
smaller  small  businesses.  These  six  were  government  websites  being  hard 
to  use,  RFP  documentation  being  too  complex,  contract  documents  being 
too  complex,  certified  cost  data  being  hard  to  provide,  past-performance 
rating  being  hard  to  obtain,  and  government  accounting  standards  being 
hard  to  meet. 

One  way  to  increase  small  business  participation  in  defense  contracts  is 
to  focus  reform  efforts  in  areas  that  small  businesses  perceive  as  barriers 
to  defense  contracts.  The  results  of  this  study  can  be  used  to  concen¬ 
trate  on  reducing  barriers  that  will  have  the  largest  effect.  A  concerted 
attempt  to  improve  communication  and  response  times  will  likely  yield 
the  best  results,  followed  by  simplifying  the  contract  proposal  process. 
Additionally,  the  creation  of  a  micro-business  category  dedicated  to 
reducing  the  factors  that  are  more  of  a  barrier  to  the  smaller  and  newer 
businesses  would  likely  increase  participation  in  defense  contracts  by 
these  “nontraditional”  small  businesses. 


Author  Note 

This  article  is  approved  for  public  release  by  the  Missile  Defense  Agency, 
16-MDA-8666  (13  May  16). 
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